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Abstract 

The field of embodied intelligence emphasises the importance of the morphology and environment with 
respect to the behaviour of a cognitive system. The contribution of the morphology to the behaviour, 
commonly known as morphological computation^ is well-recognised in this community. We believe that the 
field would benefit from a formalisation of this concept as we would like to ask how much the morphol- 
ogy and the environment contribute to an embodied agent's behaviour, or how an embodied agent can 
maximise the exploitation of its morphology within its environment. In this work we derive two concepts 
of measuring morphological computation, and we discuss their relation to the Information Bottleneck 
Method. The first concepts asks how much the world contributes to the overall behaviour and the second 
concept asks how much the agent's action contributes to a behaviour. Various measures are derived from 
the concepts and validated in two experiments which highlight their strengths and weaknesses. 

Keywords. Information Bottleneck Method; Embodied Artificial Intelligence; Morphological Computa- 
tion; Information Theory; Quantification; Measure; Sensori-Motor Loop 

1 Introduction 

Morphological computation is discussed in various contexts, such as DNA computing, and self-assembly [see 
? ? , for an overview]. This work is concerned with morphological computation in the field of embodied 
intelligence. In this context it is often described as the trade-off between morphology and control [? ], which 
means that a well-chosen morphology can reduce the amount of required control substantially. Hereby, a 
morphology refers to the body of a system, explicitly including all its physiological and physical properties 
(shape, sensors, actuators, physical properties, etc.) [? ]. The consensus is that morphological computation 
is the contribution of the morphology and environment to the behaviour, that cannot be assigned to a 
nervous system or a controller. The following quote very nicely describes how the shape of an insect wing 
in flight is not entirely determined by the muscular system, but by the interaction of the wings' morphology 
with the environment: 

However, active muscular forces cannot entirely control the wing shape in flight. They can 
only interact dynamically with the aerodynamic and inertial forces that the wings experience 
and with the wing^s own elasticity; the instantaneous results of these interactions are essentially 
determined by the architecture of the wing itself: its plan form and relief, the distribution and 
local mechanical properties of the veins, the local thickness and properties of the membrane, 
the position and form of lines of flexion. The interpretation of these characters is the core of 
functional wing morphology. [? , see p. 188] 

The last sentence of this quote nicely summaries that the function of the wing is determined by the interaction 
of the environment with the physical properties of the wing. It is the quantification of this sort of contribution 
of the morphology to the behaviour of a system that is in the focus of this work. The difference to previous 
literature by ? ] and ? ], who investigated morphological with respect to either the actuation (Paul) or the 



sensors (Lundh) only, is that we will measure morphological computation of embodied agents acting in the 
sensori-motor loop, including both, sensors and actuators. 

To understand which aspects a measure must cover and which it should omit, this paragraph will 
discuss two different artificial systems which show morphological computation. To most vivid example of 
morphological computation is the Passive Dynamic Walker by ? ]. In this example, a two-legged walking 
machine preforms a naturally appealing walking behaviour, without any need of control, as a result of a well- 
chosen morphology and environment. There is simply no computation available, and the walking behaviour 
is the result of the gravity, the slope of the ground and the specifics of the mechanical construction (weight 
& length of the body parts, deviation of the joints, etc.). If any parameter of the mechanics (morphology) or 
the slope (environment) are changed, the walking behaviour will not persist. Hence, we will only investigate 
morphological computation as an effect that emerges from the interaction of the control system, the body 
and the environment, also known as the sensori-motor loop (see next section). The Passive Dynamic Walker 
is, in this context, understood as an embodied agent without actuation. 

One may argue that the Passive Dynamic Walker is a purely mechanical system, and that speaking 
of morphological computation is an overstatement in this case. This is a valid point of view, as purely 
mechanical system do not perform calculations as we intuitively understand the term. Nevertheless, we 
claim that it is not an overstatement in the case of the Passive Dynamic Walker, as this system was explicitly 
built to simulate the morphological computation that is present in the human walking behaviour. For this 
purpose, a mechanical system was constructed that reflects the morphological properties of the lower half 
of a human body as far as it is required to understand and model the principles involved in human walking. 
Therefore, the Passive Dynamic Walker is allegoric for morphological computation present in the locomotion 
of humans. 

A second impressive example for morphological computation is BigDog [? ]. This robot is a four- 
legged walking machine, which is built as a companion for humans operating in the field. The outstanding 
feature, with respect to this paper, is its morphological design. Instead of classical electric motors, BigDog 
has hydraulic actuators, which enable it to handle situations in which other walking machines would fail 
and even suffer severe damage. Most impressive is the video, which shows how BigDog handles a very 
slippery grouncQ Watching the video (see footnote below), it seems impossible to program such a balancing 
behaviour for an arbitrarily slippery ground. That BigDog is nevertheless able to cope with it, is also the 
result of the hydraulic actuators and the well-chosen morphological design in general. 

The next step is to analyze, what both examples have in common, and in which aspects the two applica- 
tions differ. The first obvious difference is the amount of available computation. Morphological computation 
is most often associated with low computational power that leads to sophisticated behaviours [e.g. ? ]. This 
intuitive understanding is well-reflected in the case of BigDog, as the on-board computation accounts for 
only a fraction of BigDog's ability to cope with a slippery ground. This intuitive understanding is not well- 
reflected in the case of the Passive Dynamic Walker, as there is no computation available at all. Therefore, 
measuring morphological computation in relation to the computational complexity of the controller does 
not seem reasonable, as the amount of computation used to generate the action does influence the amount 
of computation conducted by the morphology. 

There is something that both examples have in common, and which is well-suited for a measure of 
morphological computation. In both cases, there is an observed (coordinated) behaviour of the system that 
is not assigned to any control. In the case of the Passive Dynamic Walker, this is obviously the case. In 
the case of BigDog, it was stated above, that the behaviour of the robot on slippery ground is not fully 
assigned to the control, but also assigned to the design of the robot, and especially its hydraulic actuation. 
Therefore, a quantification should capture how much of the behaviour is assigned to the morphology and 
environment, and not to the controller. 

This work is organised in the following way. The next section (see Sec. [2]) describes the sensori-motor 
loop and it representation as a causal graph, and thereby, presents the conceptual foundation that is required 
for the remainder of this work. In the third section (see Sec. [s]) we will then derive the different measures 
for morphological computation. The section begins with a summary of the intuitive understanding of 
morphological computation, which is then followed by a discussion of two concepts to measure it. The 



^ Watch http://www.youtube.com/watch?v=WlczBciiXlWw at about Imin 24s 
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(a) Schematics of the sensori-motor (b) Representation of the sensori-motor loop as a causal 

loop graph 

Figure 1: Schematics and causal graph of the sensori-motor loop. The figure (a) shows the conceptual 
understanding of the sensori-motor loop. A cognitive system consists of a controller, a sensory and actuator 
system, an embodiment and it is situated in an environment. The basic understanding is that the controller 
sends signals to the actuators which affect the environment. Information about the environment and also 
about internal states are sensed by the sensors, and the loop is closed when this information is passed to 
the controller. The figure (b) shows the representation of the sensori-motor loop as a causal graph. Here 
wt represents the world state at time t. The world is everything that is physical, i.e. the environment and 
the morphology. The variables st and at are the signals provided by the sensors or passed to the actuators, 
respectively. They are not to be mistaken with the sensors and actuators, which are part of the morphology, 
and hence, part of the world. 



section ends with relating them to the Information Bottleneck Method by ? ]. Two experiments to validate 
and analyse the derived measures are conducted in the fourth section (see Sec. The fifth section (see 
Sec. [5]) discusses the results and the final section (see Sec.|6]) concludes this work. All calculations are carried 
out in the appendix. 



2 Sensori-Motor Loop 

To derive a quantification of morphological computation of an embodied system, it requires a formal model 



of the sensori-motor loop. This will be briefly summarised in the following paragraph (see also Fig. 1(a)). 
A cognitive system consists of a brain or controller, which sends signals to the system's actuators. The 
actuators affect the system's environment. We prefer the notion of the system's Umwelt [? ], which is the 
part of the system's environment that can be affected by the system, and which itself affects the system. 
The state of the actuators and the Umwelt are not directly accessible to the cognitive system, but the 
loop is closed as information about both, the Umwelt and the actuators are provided to the controller by 
the system's sensors. In addition to this general concept, which is widely used in the embodied artificial 
intelligence community [see e.g. ? ] we introduce the notion of world to the sensori-motor loop, and by that 
we mean the system's morphology and the system's Umwelt. We can now distinguish between the intrinsic 
and extrinsic perspective in this context. The world is everything that is extrinsic from the perspective of 
the cognitive system, whereas the controller, sensor and actuator signals are intrinsic to the system. 

The distinction between intrinsic and extrinsic is also captured in the representation of the sensori-motor 
loop as a causal or Bayes'ian graph (see Fig. l(b)[ ). The random variables C, A, VF, and S refer to the 



controller, actuator signals, world and sensor signals, and the directed edges reflect causal dependencies 
between the random variables. Everything that is extrinsic in captured in the variable VF, whereas S, C, 
and A are intrinsic to the system. The random variables S and A are not to be mistaken with the sensors 
and actuators. The variable S is the output of the sensors, which is available to the controller or brain, the 
action A is the input that the actuators take. Consider an artificial robotic system as an example. Then 
the sensor state St could be the pixel matrix delivered by some camera sensor and the action At could be a 
numerical value that is taken by some motor controller to be converted in currents to drive a motor. 
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Figure 2: Outline of the third section of this work, in which the measures for morphological computation 
are derived. The first step is to provide a description of morphological computation. From this description 
two concepts are presented in the second step, which are then formalised in the third step. The last step 
adapts the formalisations to measure which operate on intrinsically available information only. 

Throughout this work, we use capital letter (X, y , . . . ) to denote random variables, non-capital letter 
(x, . . . ) to denote a specific value that a random variable can take, and calligraphic letters (Af, 3^, 
. . . ) to denote the alphabet for the random variables. This means that xt is the specific value that the 
random variable X can take a time t G N, and it is from the set xt ^ X . Greek letter refer to generative 
kernels, i.e. kernels which describe an actual underlying mechanism or a causal relation between two random 
variables. In the causal graphs throughout this paper, these kernels are represented by direct connections 
between the corresponding nodes. This notation is used to distinguish generative kernels from others, such 
as the conditional probability of given that ct was previously seen, denoted by ^(5^1^), which can be 
calculated or sampled, but which does not reflect a direct causal relation between the two random variables 

and 5^+1 (see Fig.[T(b)]). 

We abbreviate the random variables for better comprehension in the remainder of this work, as all 
measures consider random variables of consecutive time indices. Therefore, we use the following notation. 
Random variables without any time index refer to time index t and hyphened variables to time index t + 1. 
The two variables W, W refer to Wt and W^+i. 

We have now defined the conceptual framework in which we will derive the measures for morphological 
computation. This is done in the next section, which starts with an overview. 

3 Measuring Morphological Computation 

We will derive the measures for morphological computation in four steps (see Fig. [2]): 

1. Descriptive definition of morphological computation 

2. Framing of two concepts which follow from step 1 

3. Formal definition of the two concepts given in step 2 

4. Intrinsic adaptations of the definitions given in step 3 

The fourth step is required because the two definitions given in the third step require information about 
the world state VF, which is generally not available. This will be discussed in detail later in this section. 

3.1 Morphological Computation 

We understand morphological computation as the contribution of the morphology and environment to the 
overall behaviour of an embodied system. This implies that morphological computation must be studied in 
the sensori- motor loop, as it is the result of the interaction of the controller, morphology and environment. 
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Measuring morphological computation means that an observer has recorded a behaviour of interest of 
an embodied agent and post-hoc asks the question how much morphological computation was present in the 
recoded sequence. 

Measuring morphological computation does not mean to capture the complexity or the quality of a 
behaviour. This are aspects of the behaviour of cognitive systems, which are not handled in our measures. 



3.2 Concepts of measuring morphological computation 

In the following, we assume that an embodied agent is well-captured by the sensori-motor loop as it was 
presented in the previous section (see Fig. [T]). Explicitly, we will use the term world as it was defined above, 
as the system's morphology and Umwelt. 

Generally speaking, morphological computation is the effect of the current world state W on the next 
world state W\ which is not assignable to the action A. Given the sensori-motor loop as we have defined 
it above (see Fig. [T]), this cannot be measured in isolation, as there is a path from W to which goes 
through the agent. Hence, we need to measure the effect of the action A on the next world state so 
that we are able to deduce the effect of W on that does not include the pathway over A. This has two 
implications. First, the resulting value of such a measure is not an absolute value per se. In fact, it can 
only be measured as the relation of the two effects W W and A W' . Second, there are two ways of 
comparing the effects. 

The first method of measuring morphological computation assumes that the next world state is 
only determined by the current world state VF, which is equivalent to assuming maximal morphological 
computation. Any measured effect of the action A on the next world state W' displays itself in a reduction 
of the resulting measurement. This is summarised in the following concept: 

Concept 1 (Negative effect of the action) Given a behaviour of interest of an embodied system, the 
amount of morphological computation is inversely proportional to the contribution of the actions of the 
system to the overall behaviour. 

The second method of measuring morphological computation assumes that the next world state W' is 
only determined by the current action A, which is equivalent to assuming zero morphological computation. 
Any measured effect of the current world state W on the next world state W' displays itself in an increase 
of the resulting measurement. This is summarised in the following concept: 

Concept 2 (Positive effect of the world) Given a behaviour of interest of an embodied system, the 
amount of morphological computation is proportional to the contribution of the world to the overall be- 
haviour. 



3.3 Formalising the concepts 



The next step is to formalise the two concepts presented above. The causal diagram (see Fig. |l(b) ) shows 



that the world kernel a{w'\w^a) — p{w'\w^a) captures the influence of W and A on W' . Therefore, it is 
the basis for our further considerations. If the action A has no effect on the next world state W' ^ then 
the world kernel reduces to p{w'\w^a) — p{w^\w) and we would state that the system shows maximal 
morphological computation. Analogously, if the world state W has no influence on the next world state W' ^ 
i.e. p{w'\w^a) — p{w\a)^ we would state that the system shows no morphological computation. Measuring 
the differences of the world kernel to the two conditional probability distributions p{w'\w) and p{w^\a) leads 
us to the formalisations of the two concepts discussed above. The Kullback-Leibler divergence measures the 
differences of two probability distributions [? ] and applying it to our scenario gives us the definitions for 
the two concepts. Their formalisations are given below, such that the value zero refers to no morphological 
computation and one refers to maximal morphological computation. 

Definition 1 (Morphological Computation as negative effect of the action) 

Let the random variables A, VF, denote the action, the current and the next world state of an embodied 
agent, which is described by the causal diagram shown in Figure \T(b^ The quantification of the morphological 
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computation as the negative effect of the action on the behaviour is then defined as 



MCa := 1 



ln|W| 



In W 



D{p{w'\w, a)\\p{w'\w)) 



( 



p{w'\w^ a) 



I 



Definition 2 (Morphological Computation as positive effect of the world) 

Let the random variables A, W, W denote the action, the current and the next world state of an embodied 
agent, which is described by the causal diagram shown in Figure \T(b^ The quantification of the morphological 
computation as the positive effect of the world on the behaviour is then defined as 



MCw 



ln|W| 



1 

InlWl 



D{p{w'\w^ a)\\p{w'\a)) 



( 



p{w^ 


a) 


p{w' 


a) 



J 



Both definitions given above require full access to the world states W and W' . This is undesired for 
two reasons. First, it limits the applicability of the measure to systems of low complexity which live in the 
domain of simple grid- world environments. This contradicts our interest in presenting measures that can 
be used to analyse natural or non-trivial artificial cognitive systems. Second, we believe that the measures 
derived here, may model an intrinsic driving force in the context of guided self-organisation of embodied 
systems. Hence, we require that any measure must operate on intrinsically available information only. The 
resulting intrinsic measures are presented below, in the order of the concepts they relate too. The next 
sections list the four measures without discussing them in detail. They are evaluated and discussed in the 
subsequent sections. 



3.3.1 Concept 1, Associative Measure 

The first measure that operates on intrinsic information only is the canonical adaptation of the first definition 
to the intrinsic perspective. It must be mentioned here, that the sensor states S and are understood as 
the intrinsically available information about the external world. Hence, the conditional probability p{s^\s) 
refers to the intrinsically available information about p{w^\w). 

Definition 3 (Associative measure of the negative eff*ect of the action) 

Let the random variables A, 5, denote the action, the current and the next sensor state of an embodied 
agent, which is described by the causal diagram shown in Figure \T(b^ The quantification of the morphological 
computation as an associative measure of the negative effect of the action is then defined as: 

ASOCa := 1 - ^^D{p{s'\s,a)\W\s)) (1) 

= 1- p{s,s,a)\ii 

' ' s.s'^S 
aeA 

The Kullback-Leibler divergence used in the definition of ASOCa (see Def. [s]) is also know as the transfer 
entropy of A on. S [? ] and it was investigated in the context to quantify the informational structure of 
sensory and motor data of embodied agents [? ]. 



p{s 



5, a 



p{s'\s) 



(2) 
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(a) Schematics of the sensori-motor 
loop for a reactive system 



(b) Representation of the sensori-motor loop for a reactive 
system as a causal graph 



Figure 3: Reactive system. In the context of this work, a reactive system is defined by a direct couphng of 
the sensors and actuators. There is no form of memory present in the system. 





(a) CIF{S S') 



(b) CIF{A S') 




(c) CIF{S S'\A) 



Figure 4: Visuahsation of the causal measure Ca- The causal graph used in the Figures (a), (b), and (c) is 
the reduction of sensori-motor loop shown in Figure [3(b) [ to two consecutive time steps. Figure (a) shows 
that the causal information flow CIF{S S') measures all causal information from S to S' ^ including 
the information that flows over A. Figure (b) shows that the causal information flow CIF{A S') only 
captures the information flowing from A to S' . Both can be used to approximate the causal information 
from S to S' ^ that does not pass through A, denoted by CIF{S S'\A)^ as shown Figure (c). 



3.3.2 Concept 1, Causal Measure 

The second measure which follows the first concept is based on measuring the causal information flow of the 
sensor and action states 5, A on the next sensor state S' . To simplify the argumentation, we first consider 
a reactive system (see Fig. |3(b)[ ). The measure will then also be presented and discussed for non-reactive 
systems, as both cases result in different measures. The idea for this measure is captured in the Figure [4| 
Please keep in mind, that the sensor states S and S' are here understood as internally available information 
about the system's Umwelt. If we talk about the causal information flow of the current sensor state S to the 
next sensor state S\ we do mean the intrinsically detectable causal information flow of the current world 
state W to the next world state W\ and not how the sensor acts on itself. The causal information flow 
from S to S\ denoted by CIF{S S') includes the information that flows from S to S' over all pathways. 



This explicitly includes the information flow from S to S' over the action A (see Fig. 4(a)). Morphological 
computation is here understood as the causal information flow from S to S' excluding the causal information 
flowing from the action A to S' (see Fig. |4(c) ). Hence, we need to subtract the causal information flow from 



A to S', denoted by CIF{A S') (see Fig. [4(b)] ) to exclude it from CIF{S S') such that we receive 
the causal information flow that goes from S to S' without passing A, denoted by CIF{S S'\A) (see 
Fig.|4(^. 

According to the general theory [? ], we talk about identifiable causal effects if the are computable of 
observational data. We stated earlier, that we are interested in intrinsic measures. Hence, we require that 
the causal effects are identifiably from observational data that is intrinsically available to the agent. We 
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refer to this type of identifiability as intrinsically identifiable. In our previous work [? ] , we showed that the 
causal effects of S on and of A on are intrinsically identifiable by the following conditional probability 
distributions: 

p{s'\do{a)) = a)p{s) (3) 

p{s'\do{s)) = 5^p(a|s) J2 P{s'\s",^)p{s") (4) 
= J2pia\s)pis'\do{a)) (5) 

These distributions need to be explained. The notation p{x\do{y)) (also denoted by p{x\y) refers to prob- 
ability of measuring x when the state of Y was set by intervention to y [? ]. Explicitly, this means that 
generally intervention is required in order to determine causation. Therefore, it is important to note, that 
the equations above (see Eq. [3] and [4]) allow us to determine the causal effects without any intervention. An 
agent can act in the sensori- motor loop, and from its observation determine e.g. the causal effect of its actions 
A on the its next sensor states [for a discussion, see ? ]. From the two probability distributions given in 
the Equations [3] and [4] we can construct the two required causal information measures for CIF{S S^) and 
CIF{A S^). The derivations for both measures are given in the appendix (see Sec. |A.1[ ). The resulting 
equations are given by 



CIF{S S') = 5^p(5) J2 P{s'\Ms)) In 
CIF{A ^S')^Y^ p{a) J2 p{s\do{a)) In 



p{s'\do{s)) 



p{s'\do{a)) 



(6) 
(7) 



The difference CIF{S S') — CIF{A S') is always negative (see Eq. 57 in Sec. A.l), and hence, the 
resulting measure is given by the following definition. 

Definition 4 (Causal measure of the negative eff*ect of the action for a reactive system) 

Let the random variables A, 5, S' denote the action, the current and the next sensor state of a reactive 
embodied agent, which is described by the causal diagram shown in Figure \l(b)\ The quantification of the 
morphological computation as a causal measure of the negative effect of the action is then defined as: 

Ca := 1 + ^^ACIF{S ^ S') - CIF{A ^ ^0) (8) 
in \ o I 

= 1 - ^D{p{s'\do{a))Ms'\do{s))) (9) 



= 1 



5: p(.,a)5:p(/|do(a))ln§f;{^ (10) 



In 151 ^ ^^^^ I ^ pis'\dois)) 



The causal information flow CIF{S S') is so far not shown to be intrinsically identifiable for non- 
reactive systems [? ]. Therefore, we consider the causal information flow from the internal controller state C 
to the next sensor state S' (see Fig. |l(b)'[ ). This is legit, because C represents the entire history of the system, 
and therefore, also the internal representation of the entire history of the world. All further calculations are 
analogous to the previous case [see also ? , for a discussion] and lead to the following definition. 

Definition 5 (Causal measure of the negative eff*ect of the action for a non-reactive system) 

Let the random variables A, C, S' denote the action, the controller and the next sensor state of a non- 
reactive or deliberative embodied agent, which is described by the causal diagram shown in Figure \T(b) . The 



quantification of the morphological computation as a causal measure of the negative effect of the action is 
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then defined as: 



:= 1 + ^_(CIF(C ^ S') - CIF{A S')) (11) 
In t> 

= 1 - j^D(p{/|do(<.))|b(/|do(<;))) (12) 

The last definition (see Def. [s]) is given for the reason of completeness only. Morphological computation 
is mainly discussed in the context of behaviours which are well-modelled as reactive behaviours (e.g. lo- 
comotion). To the best of our knowledge, it has so far not been discussed in the context of non-reactive 
or deliberative behaviours. Therefore, the first definition of the causal measure (see Def. ^ suffices for all 
currently discussed cases of morphological computation in the field of embodied artificial intelligence. 

This concludes the measures of the first concept, in which morphological computation is calculated 
inversely proportional to the influence the action A has on the next world state VF^ The next section 
discusses the measures for morphological computation in which it is calculated proportionally to the effect 
the world has on itself. 



3.3.3 Concept 2, Associative Measure 

Analogous to the associative measure of the first concept, the associative measure of the second concept 
adapts the Definition [2] to the intrinsic perspective by replacing W hj S and by S^. This leads to the 
following definition. 

Definition 6 (Associative measure of the positive effect of the world) 

Let the random variables A, S, denote the action, the current and the next sensor state of an embodied 
agent, which is described by the causal diagram shown in Figure \T(b^ The quantification of the morphological 
computation as an associative measure of the positive effect of the world is then defined as: 

ASOCw := ^^D{p{s'\s,a)Ms'\a)) (14) 

The next measure approximates the dependence of the next world state W' on the current world state 
W solely based on the internal world model p{s'\s^ a). 



p{s'\s, a) 
p(s^\a) 



(15) 



3.3.4 Concept 2, Conditional Independence 

The idea of this measure is to calculate how much the conditional probability distribution p{s^\s) estimated 
from the recoded data differs from the assumption that the world did not have any effect on itself, denoted 
by the conditional probability distribution p{s^\s). The Figure [s] shows the difference between the two 
conditional probability distributions graphically. The following equations show how they can be calculated 
from the observed data. 

P{s\s) = ^p{s\s,a)p{a\s) (16) 

P(s'\s) = '^p{s'\a)p(a\s) = ^p{a\s) ^ p{s'\s'\ a)p{a\s'')^^^ (17) 
aeA aeA s'^eS ^^^^ 

The measure is then defined as the Kullback-Leibler divergence of p{s^\s) and p{s^\s). 
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w ► w w w 

V / \ \ / \ 

S ► A S ► A 

^ , , ^ 

p{s'\s) = J2 P(s'|s, a)p(a|s) ~p{s'\s) = ^ p(s'|a)p(a|s) 



Figure 5: Visualisation of the conditional independence measure Cw- The left-hand side shows how the 
conditional probability distributions p{s'\s) can be calculated from the world model p(s'\s^ a) and the policy 
p{a\s). The right-hand side shows how p{s^\s) changes, if one assumes that the world does not influence itself 
(gray arrow between W and VF'), and if this is reflected in the internal world model (gray arrow between S 
and S^). The difference of both measures the morphological computation. 

Definition 7 (Conditional dependence of the world on itself) 

Let the random variables A, S, denote the action, the current and the next sensor state of an embodied 
agent, which is described by the causal diagram shown in Figure \T(b^ The quantification of the morphological 
computation as the error of the assumption, that the next world state is conditionally independent of the 
previous world state is then defined as: 

Cw --^ v^Mp{s\s)\\p{s'\s)) (18) 



In 151 



In 15 



p{s'\s) 
p{s'\s) 



(19) 



This concludes the presentation of definitions of morphological computation measures. The next section 
discusses their relation to the Information Bottleneck Method [? ] . 

3.4 Relation to the Information Bottleneck Method 



W W X Y 



(a) Concept of morphological (b) Concept of the Information 
computation Bottleneck Method 

Figure 6: Visualisation of the relation of the concepts to the Information Bottleneck Method. Figure (a) 
shows the general concept of morphological computation. The world states are highly correlated (red arrow), 
whereas the world and action are only weakly correlated (blue arrow). The Figure (b) show the concept of 
the Information Bottleneck Method. The difference is that a strong correlation between the action A and 
and a weak correlation between W and A are required. For a discussion, please read the text below. 

We will discuss the relation of morphological computation to the Information Bottleneck Method by ? ] 
along with the two graphs shown in Figure [oj The graph on the left-hand side (see Fig. |6(a) ) shows how we 



understand morphological computation. It is present, if the current world state W has a high influence on 
the next world state W' (denoted by a red arrow) and the current action A has a low influence on the next 
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world state (denoted by a blue arrow). Hereby, it is not relevant how much the world has influenced the 
action (denoted by a black arrow) . 

The Information Bottleneck Method is visualised in the Figure |6(b)[ Here, the influence of the world 
on the action is minimised and the influence o f the action on the next world state is maximised. The 
action A is the bottleneck variable X (see Fig. 6(b)) for the relevant information that is passed through 
the world. This appears connected for the following reason. Conceptionally speaking, reducing the required 
alphabet in A to still maintain the information the world carries about itself is related to the notion of 
morphological computation as it means that only a few actions should be chosen by the agent to maintain 
the behaviour. Formally, the two concepts (Information Bottleneck and morphological computation) are not 
easily aligned. The Information Bottleneck Method requires to minimised the Kullback-Leibler divergence 
D{p(w^\w)\\p{w^\a)) [see e.g. Eq. (27) in ? ], whereas morphological computation is deflned to minimise 
the Kullback-Leibler divergence D{p{w^\w,a)\\p{w^\w)) (see Def. [T]) or to maximise the Kullback-Leibler 
divergence D{p{w^\w,a)\\p{w^\a)) (see Def. [2]). We could currently not solve this contradiction, but we 
believe that it should be followed, as it could unify the presented concepts MCa and MCw- 

This concludes the presentation and discussion of the two concepts, their formalisations, and the resulting 
four measures that rely on internal variables only. The next step in this work is to evaluate how these different 
measure behave, when they are applied to experiments. This is done in the next section. 



4 Experiments 

This section applies the morphological computation measures to two different experiments. The flrst ex- 
periment is a simplifled parameterisable model of the one-step sensori-motor loop for reactive systems (see 



Fig. 3(b)). The model is deflned by the four transition maps acf)^^{w'\w, a), (3(^{s\w), 7r^(a|5), a,iid Pr{w). For 
each map, the indices refer to its parameters. Hence, the entire model is parameterised by flve parameters, 
of which two are kept constant in the experiments below. 

The second experiment is designed as a minimal physical system, that allows a transition between the 
Dynamic Walker [? ] and a classically controlled humanoid. 



4.1 Binary Model Experiment 

In the introduction to this paper, it was stated that the measures presented in this work should be applicable 
to biological systems. In the previous section (see Sec.js]), it was stated that the measures should not operate 
on the world state also because it is not accessible for any example other then simple toy worlds. This 
section now evaluates the two concepts and four measures based on such a simple toy world example. This 
might appear as a contradiction to the previous statements, and therefore, the application of the measures 
to this model needs a further explanation. 

In order to understand how the measures differ and in which aspects they are alike, it is best to analyse 
them under a fully controllable setting. Using real robots only allows to fully control a very limited set of 
parameters of the entire system, namely the parameters of the policy and partially those of the embodi- 
ment. This improves only slightly in simulated robotics, depending on the chosen implementation. These 
uncontrollable or hidden parameters require robustness of the applied methods, and therefore, are the main 
argument favouring virtual or real robots over grid world environments. The goal of this section is different. 
Here, we want to validate the measures by controlling the influence of the world on itself in addition to con- 
trolling the systems parameters. Therefore, it is necessary to choose an experiment which allows us to fully 
control every aspect of the sensori-motor loop, explicitly including the world transition kernel a(w^\a,w). 
The next section will then validate the measures in more realistic experiment. 

The minimalistic model discussed in this section is shown in Figure [7| and deflne by the following set of 
equations: 
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Figure 7: Visualisation of the binary model. This figure shows the causal graph that is used in the binary 
model experiment (see text below). It is the graph representing the single step sensori-motor loop of a 
reactive system, where the indices of the transition maps refer to their parametrisation. 



^4>A^>,a)-^ (20) '''^^''''^^Ea'eQ^^"'^ ^""^ 

Ms\w) = ^ ^ (21) = ^^rw'^ (23) 

where all random variables are from the same binary alphabet, i.e. w' ^ a, 5, 5' G = {"I? !}• The model 
is parametrisable by the variables 0, '0, fJi^r G M^, which control how deterministic the kernels are. If we 



consider the policy 7r^(a|5) which is controlled by the parameter ^, then we see from Equation 22 that ji — 
results in TT^{a\s) — 1/2 for all a,5 G f^. The policy is completely random, because both actions a—1 and 
a = — 1 occur with equal probability, independent of the current sensor value 5. On the other hand, if we 
set /i » 0, then the policy changes to a Dirac measure on the sensor and actuator states, i.e. 7r^(a|5) = 5as^ 
where 5xy — 1 \i x — y and zero otherwise. The parameter allows a linear transition between these two 
cases. 

For simplicity, but without loss of generality, the following two assumptions are made. First, it is assumed 
that all world states w ^ occur with equal probability, i.e. p{w = 1) = p{w = — 1) = 1/2. Furthermore, 
we assume a deterministic sensor, i.e. » 1 ^ p{s\w) — 5swi which means that the sensor is a copy of the 
world state. The first assumption does not violate the generality, because it only assures that the world 
state itself does not already encode some structure, which is propagated through the sensori-motor loop. 
Second, in a reactive system, as it is shown in Figure [7| the sensor state S and A could be reduced to a 
common state, with a new generative kernel ^{a\w) — Ti{a\s) o P[s\w). Hence, keeping one of the two kernels 
deterministic and varying the other in the experiments below, does not effect the generalisation properties 
of this model. This leaves three open parameters '0, 0, and /x, against which the morphological computation 
measures are validated (see Figures |9, 10, and 11). 



To be able to understand if the plots validate the measures, it is necessary to understand how the 
parameters affect the morphological computation in this model. To simplify the argumentation, it is first 
assumed that the policy is completely random (/x = 0), and hence, the action A is independent of the 
previous world state W (this assumption is dropped below). What follows is that the world transition 



kernel a{w'\a^w) (see Eq. 35) can be divided into four cases (see Fig.|8]), which are classified by the effect of 
(j) and '0 on the morphological computation. To understand how the parameters affect the measure, they are 
first discussed with respect to the world kernel a{w'\w^ a). This is then related to how the Kullback-Leibler 
divergences used in the two definitions (see Def. [l]and Def. [2]) are affected. The analysis is conducted with 
respect to the two concepts MCa and MCw will it starts with the two unambiguous cases. The intrinsic 
measures are discussed later in this section. 

1st Case: high, '0^0. This case refers to a world, which is only influenced by the last world 

state, as the world kernel a{w'\a^w) reduces to a Dirac measure on the current and next world state (see 
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MC^t/ MC^T 



MCAi/MCwi 



MC^t/MCwi 



Figure 8: Visualisation of the four discussed cases of the world model. High values are indicated by an 
arrow pointing upwards (t), and low values with an arrow pointing downwards (J,), i.e. (/> 4 refers to the 
case, where (f) ^ and </> ?a 0, resulting in a high influence of W ^ W and low influence of ^ — >• W'. The 
figures are discussed in the text below. 



Fig-i): 



a(w^\w^ a) = S^f^ 



(24) 



The result is a high measured morphological computation for both measures MCa and MCw, as p{w^\w, a) = 
p(w'\w)^ and hence, —D{p{w^\w^a)\\p(w'\w)) and D{p{w^\w^ a)\\p{w'\a)) are both maximal. 



2nd Case: ^ 0, '0 » 0. This case refers to a world, which is only influenced by the last action, as the 
world kernel a{w^\w, a) reduces to a Dirac measure on the action and next world state (see Fig. [s]): 

a{w^\w,a) S^^a- (25) 

The result is a low measured morphological computation for both measures MCa and MCw, as p{w'\w, a) = 
p{w^\a), and hence, —D(p{w^\w,a)\\p{w^\w)) and D(p(w^\w,a)\\p(w^\a)) are both minimal. 



/i =0.0 



0.5 




MC 
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/i= 1.0 
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/i = 2.0 



0, 
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Figure 9: Numerical results for the measures MCa and MCw and the binary model. The three figures in 
the first row shows the results for MCa, which increasingly deterministic policy tt (from left to right). The 
second row shows the results for MCw, also with increasingly deterministic policy. For all plots, the base 
axes are given by world kernel parameters tjj and (f). The plots confirm the considerations discussed in the 
text below. 



3rd Case: (/) ^ 0, '0 ^ 0. This case refers to a world, in which the next world state is independent of the 
previous world state and the previous action, which is described by 

a{w'\w,a) = 1/2- (26) 
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The two measures MCa and MCw give different results, because the two Kullback-Leibler —D{p{w'\w, a) \ \p{w'\ 
and D{p{w'\w, a)\\p{w'\a)) are both zero. Consequently, the two measures lead to MCa — 1 and MCw — 0. 

4th Case: </> ^ 0, V 3> 0. This case is a mixture of all previous cases, as the world behaves according to 

/ /I \ — j ^w'w if = a 
1 otherwise 

The fourth case is similar to the third case if the world state W is not equivalent to the action A as then 
the next world state W is independent of both. We saw from the third case, that this leads to MCa = 1 
and MCw — 0. If the action A and the world state W are equal, the first measure reduces the amount of 
morphological computation, whereas the second measure increases the amount of morphological, leading to 
MCa < 1 and MCw > 0. 

To complete the analysis of the model before the numerical results are discussed, the assumption of 
a random policy is dropped. We now assume a fully deterministic policy {ji ^ 0). In this case, the 
policy reduces to a Dirac measure on the world and action state, as the sensor state is a copy of the world 
{I3{s\w) = Ssw)-, and therefore, 7ry^(a|s) = 5as = ^as^sw = ^aw Is also follows that p{w'\a) = p{w'\w). For the 
two measures, it follows that: 



MCa = 1 

= 1 
= 1 



ln\W 
1 

\n\W 



-D{p{w'\w, a)\\p{w'\w)) 
■D{p{w'\w)\\p{w'\w)) 



MCw 



1 



ln\W 
1 

ln\W 




■D{p{w'\w,a)\\p{w'\a)) (28) 

■D{p{w'\w)\\p{w'\w)) (29) 

(30) 



The effect, that a deterministic pohcy leads to different results for both concepts will occur in the following 
experiments. It will be discussed in detail in the next section (see Sec. [5]). 

The four cases are identical with the four corners of the plane in Figure [9| and the four cases are 
well-reflected in the plots. The plots also visualise the difference of the two concepts for the last two cases 
(0 ^ 5, '0 ^ 5 and ^ 0, '0 ^ in Fig. l9]). 



/i = 0.0 
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/i = 2.0 
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ASOCh/ 
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Figure 10: Comparison of the two measure ASOCa and ASOCw- The results confirm the measures are 
intrinsic adoptions of the measures MCa and MCw (see Figure [9]). 



Next, this section now applies the four intrinsic measures to the binary model. The results are shown 



for the variations of the three parameters /i, ^, in Figure 10 and Figure [TT 



14 



The intrinsic measures require the probabihty distributions p(s) and p{s'\s^a), which can be calculated 

(31) 



from the Equations [20| to [23| in the following way: 

p{s) ^ I3{s\w)p{ 



p{s'\s,a)= ^ p{s',w',w\s,a) = ^ 
p{s\ w\ W, 5, a) 



p(s\ w\ w, 5, a) 
p{s,a) 



7T{a\s)p{s) 



W 



7r(a 


s) 


7r(a 


s)p{s) 



(32) 

(33) 
(34) 
(35) 
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Figure 11: Comparison of the two measure Ca and Cw- The results confirm the measures are intrinsic 
adoptions of the measures MC a and MCw (see Figure [9]) . 

The results show that both measures in the first concept and both measures in the second concept 
show very similar results compared to the formalisation of the concepts (compare Fig. [9] with Fig. 10 and 
Fig. [9| with Fig. 11). This can be expected because the sensor distribution in equivalent to the world 



distribution due to the Equation [3T] and because the sensor kernel was set to be a Dirac measure on the 
world {f3{s\ — Sgw)- Nevertheless, the plots show that the intrinsic adaptations of the formalisation of the 
concepts capture what the concepts specify. 



4.2 Rotator 

The previous section verified the measures in a very simple, yet illustrative binary model of the sensori-motor 
loop. It was shown that all measures produce the desired output for variations on the transition probabilities 
of the world and the policy. This section applies the intrinsic measures to an experiment that is inspired by 
common examples of previously discussed systems that show high and low morphological computation. 

The Dynamic Walker [? ] is designed to emulate the natural walking of humans. One characteristic is 
that only half of the leg movement involved in the walking behaviour is actively controlled by the brain [? 
]. It is the stance phase, i.e. the time during which the foot touches the ground and moves the rest of the 
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body forwards, which is fuhy controlled. The swing phase, i.e. the time in which the leg swings forward 
before another stance is initiated, is only partially controlled. One may say that the body lets loose and 
gravity takes over. This is exactly what the Passive Dynamic Walker highlights, as it only exploits its body 
and the environment, i.e. the slope and gravity to produce a natural walking behaviour. 

The world's most advanced humanoid robotj^Asimo is an example of a system, that does not show any 
morphological computation as it is discussed in this work. The trajectory of each part of the morphology is 
carefully controlled during the stance and swing phase at all times. This is also the reason, why the motion 
of Asimo does not appear natural although it is very smooth. 

We are now presenting a simple experiment, that allows us to vary the amount of exposed morphological 
computation between the two examples discussed above. For this purpose we chose a pendulum, which can 
rotate freely around its anchor point. The task of our controller is to consistently rotate the pendulum 
clockwise. The Asimo case is approximately given, if the angular velocity of the pendulum is controlled 
at every instance in time, whereas the Dynamic Walker case is approximated if the angular velocity of the 
pendulum is only altered for large deviations of the current angular velocity from the target velocity. This 
will be more clear, after the equations have been presented. The pendulum is modelled by the following 
equation: 



= mW{t)+-fW(t)+mgsm(9{t)) - f{t), 



(36) 



where f{t) is the force imposed by the controller (see below), 7 is the friction coefficient, 9{t) is the current 
angle of the pendulum, m is its point mass which is located from the center by the length /. The Equation 
[36] was numerally solved for t — O.O5 . . . O.OI5 using NDSolve method of Mathematica 8.0 [? ]. The actions 
f{t) were modified only for t — 0.01, 0.02, . . . , T, and kept constant while the Equation 36 was solved. This 



refers to a behaviour update frequency of lOOHz. The controller is defined by the following set of equations: 



s{t) = d{t) + u{-ri,i^) 
g{t)^ e - s{t) - 



sign{s{t)) P + sign{s{t))F^ 



m 



1. measured error 
+1 

9 it) 



-1 







2. reduction due to bias 3. minimial force factor 

</3 ' 



-f^max if 



if 



e - s{t) 
e - s{t) 



(37) 
(38) 

(39) 



The function of the controller is explained along the three equations Eq. 37 to Eq. |39j The first Equation (see 
Eq. 37) adds uniformly distributed noise u(—r]^ 77) to the sensed angular velocity 0(t), which is then presented 
as sensor value s{t) to the controller. It was discussed in the previous section that a deterministic reactive 
system prevents the possibility to distinguish between the information flow of A ^ W' and W W as A 
becomes a determinism function of VF, and hence, A and W are equivalent with respect to our measures. 
Therefore, noise is added to the sensors, such that St is not deterministically dependent on Wt- The Equation 
38) determines the strength of the response of the controller as a function of the difference of the sensor 
value s{t) and the target angular velocity O (see first term in Eq. 38). This function is only executed if the 
difference of the sensor value to the goal is larger than a threshold value /3. Hence, to ensure sensitivity, the 



threshold value is subtracted from the absolute value of the response (see second term in Eq. 38). The third 
term in the Equation [38] ensures a minimal response strength, and hence, an minimal effect of the action At 
on the next sensor values St+i- 

All parameters shown in the Equations [36] - [39l were evaluated systematically. The most distinctive results 
were found for the following values, which is why they were chosen for presentation here: 



i^max = 10.0 

F^in = 0.25 



/3 G [0.0,2.0] 
7] G [0.0,0.5] 



e = 27r 
7 = 0.0 



m 1.0 
g = 9.81 



5000 



(40) 
(41) 



Cited from http : / / asimo . honda . coin| 
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Figure 12: Results of the intrinsic measure in the rotating pendulum experiment. From left to right ASOCa, 
Ca, ASOCw, and Cw- The plane in each plot is defined by the noise 77 and the threshold parameter /3. The 
values are averaged over 10 runs, for 7/ E {0, 0.025.0.05, . . . , 0.5} and /3 G {0, 0.01, 0.02, . . . , 2.0}. The results 
are discussed in the text below. 



Results The results are presented in two figures (see Fig. 12 and Fig. 13). The Figure 12 shows three- 



dimensional plots in which the plane is defined by the noise 77 and the threshold /3, and for which the 
height z is given by the measurements of the denoted measure. The Figure p!3| shows the transients of 

and Eq. j39) for the four 
ow the transients for the 



the sensor values s{i) (see Eq. 37) and the normalised action ^(t)|^|^ (see Eq. |38 
extremal points of the plots in Figure [T2j Therefore, the five plots in Figure [Ts] ^ 
configurations (77, /3) G {(0,0), (0.0,2.0), (0.5,0.0), (0.5,2.0)}. The discussion of the results in the following 
paragraphs refers the these two figures and follows the ordering of the transient plots in Figure [l3j All 
results were obtained by sampling the world model p(5'|5, a), policy ^(a\s) and sensor distribution ^(5) from 
the data stream of 5000 behaviour updates (see Eq. [36]). The sampling was performed according to [? ] (it 



is additionally also briefly described in section [Al2 of the appendix). The sensor values were binned in the 
interval St G [0, 8] and the actions were binned in the interval At G [—1,1] with 30 bins each. 

The first discussed configuration is (77, /3) = (0, 2.0). The transients show a clear picture, as no action of 
the controller is performed other than /(t) = after about 2 seconds. Consequently, for this configuration 
any measure should result in maximal morphological computation leading to a value close or equal to one, 
as the inertia, and hence, the world W is the only cause for the observed behaviour. We see that both 
measure of the first concept (ASOCa and Ca) deliver a value close or equal to 1 (all values in the Figure 
13 are rounded to the second decimal place and are averaged over 100 runs). The two measures of the 
second concept (ASOCw and Cw) show their maximal values for this configuration at approximately 0.54 
(maximal refers to all configurations shown in Figure 12). This again highlights the differences of the two 
concepts. The first concept reduces the amount of morphological computation by the measured effect of 
the action A on the next world state W' ^ measured through S' ^ whereas the second concept increases the 
morphological computation by the measured effect of W on W' ^ captured by S and S' . This explains, why 
both concepts show different maximal values for this configuration. 

The second discussed configuration is given by (77, /3) = (0, 0). We expect no morphological computation, 
because a threshold of /3 = means that the pendulum is controlled at every time step t, and no noise on 
the sensor values 77 = means that the action is only dependent on the actual angular velocity Qif). We 
see that in this case, both concepts lead to very different results. The second concept matches our intuition 
better, as both measures (ASOCw and Cw) deliver values close to zero. The measures in the first concept 
result in values close to one, as the action A is deterministically dependent on the world state VF, captured 
by 5, and hence, the effect ol A ^ W' is fully determined by the effect olW ^ A. Consequently, in the 
first concept, the current world state W almost fully determines the next world state W' ^ which leads to the 
high values. We to point out, that there is no preference to any one of the two results. Both seem equally 
valid at the current stage of the discussion. This is strong evidence that both concept capture important 
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aspects, but that a clear picture cannot be presented 




ASOCa: 0.99 Ca: 1.00 ASOCw: 0.54 Cw: 0.54 




ASOCa: 0.92 Ca: 0.99 ASOCw: 0.46 Cw: 0.55 



yet. The wih be discussed in detail in the next section. 




ASOCa: 0.96 Ca: 0.99 ASOCw: 0.01 Cw: 0.01 




ASOCa: 0.96 Ca: 0.97 ASOCw: 0.45 Cw: 0.51 



Figure 13: Transients. Please see the legend to each plot and read the text below for a discussion. The 
values were averaged over 100 runs. 

The other three configurations show that a noisy controller output is filtered by the inertia of the 
pendulum, resulting in an angular velocity approximating the target velocity. The amount of noise 77 
increases the measured morphological computation, as well as the higher threshold value /3. Both results 
correspond with the expectation. 



5 Discussion 



? ] state that one problem with the concept of morphological computation is that while intuitively plausible, 
it has defied serious quantification efforts. From the experiments presented in the previous section we 
can now understand where this problem rises from. In the context of deterministic world, a determinism 
embodiment, and a deterministic reactive policy one cannot distinguish between the effect of the world W 
and the effect of the action A on the next world state as the action A is given by a deterministic function 
of W. Hence, the two states can be subsumed to a single state, which results in the obtained difference of 



the measures in the two concepts (see Eq. 30). We will study this with respect to a Gedankenexperiment 
by ? ]. In his book, Braitenberg discusses different vehicles, which show different behaviours due to a very 
simple sensor motor coupling. The vehicle 3 is of such kind. It has two sensors that can detect light sources 





Figure 14: Braitenberg Vehicle 3. The Figure shows two Braitenberg vehicles, each of them is equipped with 
two light sensors and two actuators. The polarity of the sensor-motor couplings determines the behaviour. 
The Braitenberg vehicle on the left-hand side is repelled by light, whereas the Braitenberg vehicle on the 
right-hand side is attracted by light. 

and two actuators with which the system can move and steer. The sensors and motors are cross-coupled 



(see Fig. 14). Depending on the polarity of the couplings, the vehicle is either attracted or repelled from 
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a light source. For this experiment, the two measures result in MCa = 1 and MCw = 0. Which one is 
considered to be correct depends on how the policy is separated from the embodiment. One may argue, 
that the coupling and the coupling strength defines the policy. In this case, one has to conclude that there 
is low morphological computation, as the next world state is mostly determined by the policy. This favors 
the measure MCw- One may also argue that the couplings between the sensors and actuators are of such 
simplicity, that they can be considered as part of the embodiment, as the sensors signals directly feed to the 
actuators. In this case, the state of the system is fully determined by the current sensor state, and hence, 
the world state. This is in favor of the first measure MCa- 

It seems that both measures capture important aspects of morphological computation, but that a final 
answer cannot yet be given here, as the discussion about the Braitenberg vehicle 3 shows. 



6 Conclusions 

This work began with an introduction of morphological computation in the context of embodied artificial 
intelligence. It was concluded, that the complexity of the controller or brain should not be taken into 
account for a measure of morphological computation. Instead, the influence of the action of an agent 
and the influence of the last world state need to be compared. Two concepts to measure morphological 
computation were then discussed and formalised, from which several intrinsic adaptations were derived. The 
different measures were evaluated in two experiments, which showed the conceptual difficultly in measuring 
morphological computation. For any fully deterministic reactive system, the influence of the action and the 
influence of the world are not easily separated, which leads to contradicting results for the two concepts. 
As this only occurs in this very special, but common case, we propose to use the contradicting results as 
an indication that the observed behaviour may be due to such a deterministic reactive system. In this case, 
noise should be introduced to break the deterministic dependency of the action on the world. 
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A Appendix 

A.l Derivation of the Causal Measure 1 



The causal information flows used in the Section [s] (see Fig. |4(a)| and Fig. |4(b)[ ), are derived by the means 
of a mutual information measure. The mutual information of two random variables X and Y is calculated 
in the following way [? ]: 



xeXyey 



Pix,y) 



p{x)p{y)_ 
= D{p{x,y)\\p{x)p{y)) 



(42) 
(43) 



This is the Kullback-Leibler divergence of the joint distribution of X and Y with respect to the marginal 



distribution of X and Y (see Eq. 43). In the case of the causal information flow of S on S' ^ the measure 
compares the joint distribution of S' and the intervened 5, given by p{s' ^ do(5)) := p{s'\do{s))p{s)^ with the 
marginal distribution of p{s')p{s)^ where p{s') is the post-interventional distribution of (compare Eq. 45 
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and 46, below). The resulting measure is then given by: 



CIF{S ^ 50 = J2 



5;p(.)5;p(/|do(.))ln 



p{s',do{s)) 
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(44) 
(45) 
(46) 



The causal information flow of A on S', denoted by CIF{A — )■ S') is derived analogously with p{s' , do(a)) := 
p{s'\do{a))p{a): 



CIF{A ^ S')= p{s', do(a)) In 

s'eS,aeA 
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(47) 
(48) 



The Equation ^ implies 
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'e5 



p{s'\do{s)) 



j;p(a|.)p(.'|do(.)) In 

s'eS \aeA J 

s'eS \aeA / 



lZs'esPis")pWois')). 



T.s"esPis") T,aeAP((^\^)p(^'\Ma)) 
12aeAPi^\^)p(^'\Ma)) 



J2aeAP(^)Pi^'\Ma)) 

Pis'ldoja)) 
EaG^P(«)^'(«'|do(a)) 



< J^p(a|s)^p(/|do(a))ln 
^ CIF{S S') < ^p(a|5)C/F(A ^ S'). 

aeA 

Summation with respect to 5 finally yields 

CIF{S S') = ^ 50 

se5 aeA 
= ^p(a)CIF(a^ 5') 

^ CIF{S S') < CIF{A ^ S') 



(49) 
(50) 

(51) 

(52) 
(53) 

(54) 
(55) 
(56) 
(57) 



It follows, that the measure for morphological computation can be represented as KL-divergence in the 
following form: 

CIFiS^S')-CIFiA^S') = - J2 a) ^ p{s'\do(a)) In j (58) 

seS,aeA s'eS ' 

^ -D{p{s'\do{amp{s'\do{s))) (59) 

Any Kullback-Leibler divergence is non-negative [? ], i.e. D(p\\q) > 0, which means that the measure given 
above (see Eq. 58) is always negative. 
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The derivation of the morphological computation measure for non-reactive control (see Fig. |l(b)[ ) is 
analogous to the reactive control [? ]. Every occurrence of 5, 5, S is replaced by c, C, C, which leads to 



CIF{C ^ S') = 

c6j s'eS 



pis'ldoic)) 



(60) 



A. 2 Sampling world model, policy and input distribution in pendulum experiments 



The probability distributions that were required in the rotator experiment (see Sec. |4.2[ ) were obtained from 
the data series by the same sampling, that we have used in a previous publication [? ] . It is briefly discussed 
for the world model below. Its application to the policy and input distribution is straight forward [? ] . The 
sampling starts with an equal distribution and the updates the rows for which a sample was detected in the 
following way: 



p^<'>{s'\s,a) 



1 



n% + 1' 



>{s'\s,a) + 



nf, + 1 



n 



^p«-i)(s'|s,a) 



^ p«-^\s'\s,a) 



if Sjig+l — s', Sn — S, Ang+1 = a 
if Sng + l 7^ s', Sn — S, — a 

if Sng ^ s or ^ a 



(61) 
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